EU accuses TikTok of trapping users in harmful ‘rabbit holes’

The European Commission accused TikTok on Friday of failing to prevent its recommendation algorithm from drawing users into “rabbit holes” of content about sadness, self-harm and body image.

The preliminary findings that TikTok violated the EU’s Digital Services Act (DSA) come amid focus on both sides of the Atlantic on whether and how social media platforms are endangering young users. 

European officials said their findings were the first time EU regulators have directly challenged the engagement-based recommendation system of a major social media platform, and, to their knowledge, was the first enforcement action of its kind worldwide.

The investigation was directly “addressing a business model” built primarily on user engagement, a Commission official told journalists.

TikTok’s ‘For You’ feed “shows users a continuous stream of personalised videos, even when those are related to mental health struggles and body image,” stated the European Commission.

“The more a user interacts with a certain type of content, the more TikTok’s ‘For You’ feed algorithm recommends similar content. This design can be harmful when such endless and repeated exposure involves, for example, content about sadness, isolation or body image.”

Brussels said TikTok had identified some of the issues it was flagging in its own risk assessments, but that its investigators found the company had failed to properly analyze those risks and set appropriate safeguards.

The findings are not final and TikTok has a chance to review the Commission’s evidence and challenge its findings.

If the DSA violations are confirmed the company could face a fine of up to 6% of its annual global revenue. The company does not publish its global revenue, nor does its parent company ByteDance.

Rabbit holes

European officials stressed the case was not about removing content, and said investigators deliberately focused on videos that were neither illegal nor against TikTok’s own rules, which one official described as the “least harmful” category of content.

Citing trends including “what I eat in a day” and “SkinnyTok,” officials said the legal content could appear harmless in isolation but became damaging through repetition, particularly for young and vulnerable users.

It follows U.S. senators in August demanding answers from TikTok over the death of Chase Nasca, a 16-year-old who took his own life in February 2022 after viewing a stream of content related to sadness and suicide.

An internal TikTok review, reported last year by Bloomberg, revealed Nasca was part of a control group comprising about 10% of the platform’s user base in which it withheld design changes intended to prevent repeated exposure to harmful subjects.

“If you get a chain of hundreds or even thousands of videos of the same kind of content, this drives people in what we call a rabbit hole, and this has a negative impact on people's mental well-being,” a European Commission official said on Friday.

TikTok’s own tools to break the loops do not work, according to the Commission, which said the platform’s “dispersion” techniques to diversify the feed are ineffective, refreshing the feed only helps briefly, and tapping “not interested” stops working after a while.

In particular, the Commission said it wants TikTok to base its algorithmic recommendations on what users explicitly say they want rather than on implicit signals such as how long they view videos for. 

They said they spent two and a half years developing “amongst the world’s most advanced investigatory infrastructures,” but that specific testing methods were necessarily secret to prevent platforms from gaming the tools.

They said they obtained the company’s internal documents and data, and analyzed around 20,000 videos with help from psychologists, psychiatrists and experts in system design. The testing “didn't really need access to the exact algorithm. We had enough information without it,” one official said.

A spokesperson for TikTok, which has the right to check and challenge the Commission’s evidence, said: “The Commission has made a series of statements that are fundamentally inaccurate and distort our platform’s architecture and our commitment to user safety.

“We have a strong record of protecting our community while giving users direct and intuitive control over their content experience,” they added.

Friday’s findings are the third the Commission has issued against TikTok this year, following one in February challenging addictive design features and another in July regarding protections on minors’ accounts.

Officials distinguished this case as being focused on the platform’s recommendation system rather than issues related to notifications or endless scrolling. They said it went further than TikTok’s recent settlement in Alabama, which “does not go as deep” into the recommendation system.

They acknowledged other platforms similarly push users into rabbit holes and said the Commission was “looking more widely beyond TikTok,” although they declined to comment beyond TikTok.

The Commission described the case as a “scene setter” for its proposed KIDS Act, introduced in September, which would prohibit platforms from relying solely on engagement signals to recommend content to children — a provision officials said was directly inspired by its TikTok investigation.